Skip to content

Conversation

@w2yehia
Copy link
Contributor

@w2yehia w2yehia commented Nov 18, 2025

On AIX, the linker's release cadence is once per year and it doesn't backport non-critical fixes to previous releases.
We would like to get thinLTO caching accessible for current customers, so this PR adds the cache flags as cl::opt options.
We can surround the code with #ifdef AIX if the patch is unacceptable as is.

@w2yehia w2yehia added the LTO Link time optimization (regular/full LTO or ThinLTO) label Nov 18, 2025
@llvmbot
Copy link
Member

llvmbot commented Nov 18, 2025

@llvm/pr-subscribers-lto

Author: Wael Yehia (w2yehia)

Changes

On AIX, the linker's release cadence is once per year and it doesn't backport non-critical fixes to previous releases.
We would like to get thinLTO caching accessible for current customers, so this PR adds the cache flags as cl::opt options.
We can surround the code with #ifdef AIX if the patch is unacceptable as is.


Full diff: https://github.com/llvm/llvm-project/pull/168567.diff

1 Files Affected:

  • (modified) llvm/tools/lto/lto.cpp (+42)
diff --git a/llvm/tools/lto/lto.cpp b/llvm/tools/lto/lto.cpp
index 467a4da27dcd8..6efefc7977e4a 100644
--- a/llvm/tools/lto/lto.cpp
+++ b/llvm/tools/lto/lto.cpp
@@ -24,6 +24,7 @@
 #include "llvm/LTO/legacy/LTOCodeGenerator.h"
 #include "llvm/LTO/legacy/LTOModule.h"
 #include "llvm/LTO/legacy/ThinLTOCodeGenerator.h"
+#include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/TargetSelect.h"
@@ -44,6 +45,29 @@ static cl::opt<bool> EnableFreestanding(
     "lto-freestanding", cl::init(false),
     cl::desc("Enable Freestanding (disable builtins / TLI) during LTO"));
 
+static cl::opt<std::string> ThinLTOCacheDir(
+    "thinlto-cache-dir",
+    cl::desc("Experimental option, enable ThinLTO caching. Note: the cache "
+             "currently does not take the mcmodel setting into account, so you "
+             "might get false hits if different mcmodels are used in different "
+             "builds using the same cache directory."));
+
+static cl::opt<int> ThinLTOCachePruningInterval(
+    "thinlto-cache-pruning-interval", cl::init(1200),
+    cl::desc("Set ThinLTO cache pruning interval (seconds)."));
+
+static cl::opt<uint64_t> ThinLTOCacheMaxSizeBytes(
+    "thinlto-cache-max-size-bytes",
+    cl::desc("Set ThinLTO cache pruning directory maximum size in bytes."));
+
+static cl::opt<int> ThinLTOCacheMaxSizeFiles(
+    "thinlto-cache-max-size-files", cl::init(1000000),
+    cl::desc("Set ThinLTO cache pruning directory maximum number of files."));
+
+static cl::opt<unsigned> ThinLTOCacheEntryExpiration(
+    "thinlto-cache-entry-expiration", cl::init(604800) /* 1w */,
+    cl::desc("Set ThinLTO cache entry expiration time (seconds)."));
+
 #ifdef NDEBUG
 static bool VerifyByDefault = false;
 #else
@@ -543,6 +567,24 @@ thinlto_code_gen_t thinlto_create_codegen(void) {
     assert(CGOptLevelOrNone);
     CodeGen->setCodeGenOptLevel(*CGOptLevelOrNone);
   }
+  if (!ThinLTOCacheDir.empty()) {
+    auto Err = llvm::sys::fs::create_directories(ThinLTOCacheDir.getValue());
+    if (Err)
+      report_fatal_error(Twine("Unable to create thinLTO cache directory: ") +
+                         Err.message());
+    bool result;
+    Err = llvm::sys::fs::is_directory(ThinLTOCacheDir.getValue(), result);
+    if (Err || !result)
+      report_fatal_error(Twine("Unable to get status of thinLTO cache path or "
+                               "path is not a directory: ") +
+                         Err.message());
+    CodeGen->setCacheDir(ThinLTOCacheDir);
+  }
+  CodeGen->setCachePruningInterval(ThinLTOCachePruningInterval);
+  CodeGen->setCacheEntryExpiration(ThinLTOCacheEntryExpiration);
+  CodeGen->setCacheMaxSizeFiles(ThinLTOCacheMaxSizeFiles);
+  CodeGen->setCacheMaxSizeBytes(ThinLTOCacheMaxSizeBytes);
+
   return wrap(CodeGen);
 }
 

@cachemeifyoucan
Copy link
Collaborator

Can you explain why you can't add flags to linker to call the API that sets the cache directly but want to rely on cl:opt?

Also if you want to rely on it in production, please add test, even we end up running on AIX only.

@github-actions
Copy link

github-actions bot commented Nov 18, 2025

🐧 Linux x64 Test Results

  • 186372 tests passed
  • 4855 tests skipped

@w2yehia
Copy link
Contributor Author

w2yehia commented Nov 18, 2025

Can you explain why you can't add flags to linker to call the API that sets the cache directly but want to rely on cl:opt?

Technically we can teach the platform linker to call the API, but users will only get it in the next OS release which is at least a year away. We have users on existing in-service OS releases that we want to provide this functionality to, and they are stuck with the platform linker they have.

Copy link
Collaborator

@cachemeifyoucan cachemeifyoucan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not a big fan of adding this if this is only used for one year but I am also not strongly against it.

Please add a test to exercise the options.

@w2yehia
Copy link
Contributor Author

w2yehia commented Nov 19, 2025

Thanks Steven (@cachemeifyoucan ) for the review

Please add a test to exercise the options.

Since these options are added to lto.cpp which is only used to build libLTO.so, I don't think we test libLTO.so in LIT, do we?

Copy link
Collaborator

@cachemeifyoucan cachemeifyoucan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Adding a new tool to test libLTO is probably overkill for this change. SGTM

@w2yehia w2yehia merged commit c2b4e48 into llvm:main Nov 19, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

LTO Link time optimization (regular/full LTO or ThinLTO)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants